Ac 2009-2249: Experiential Learning and Strictly Proper Scoring Rules
نویسنده
چکیده
Experiential learning is perhaps the most effective way to teach. One example is the scoring procedure used for exams in some decision analysis programs. Under this grading scheme, students take a multiple-choice exam, but rather than simply marking which answer they think is correct, they must assign a probability to each possible answer. The exam is then scored with a special scoring rule, under which students’ best strategy is to avoid guessing and instead assign their true beliefs. Such a scoring function is known as a strictly proper scoring rule. In this paper, we discuss several different scoring rules and demonstrate how their use in testing situations provides insights for both students and instructors.
منابع مشابه
Choosing a Strictly Proper Scoring Rule
S proper scoring rules, including the Brier score and the logarithmic score, are standard metrics by which probability forecasters are assessed and compared. Researchers often find that one’s choice of strictly proper scoring rule has minimal impact on one’s conclusions, but this conclusion is typically drawn from a small set of popular rules. In the context of forecasting world events, we use ...
متن کاملProper Scoring Rules, Dominated Forecasts, and Coherence
De Finetti introduced the concept of coherent previsions and conditional previsions through a gambling argument and through a parallel argument based on a quadratic scoring rule. He shows that the two arguments lead to the same concept of coherence. When dealing with events only, there is a rich class of scoring rules which might be used in place of the quadratic scoring rule. We give condition...
متن کاملStrictly Proper Scoring Rules, Prediction, and Estimation
Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the forecast and on the event or value that materializes. A scoring rule is strictly proper if the forecaster maximizes the expected score for an observation drawn from the distribution F if she issues the probabilistic forecast F , rather than any G 6= F . In prediction problems, strictly prope...
متن کاملEvaluating probability forecasts in terms of refinement and strictly proper scoring rules
This note gives an easily verified necessary and sufficient condition for one probability forecaster to empirically outperform another one in terms of all strictly proper scoring rules.
متن کاملProbabilism, entropies and strictly proper scoring rules
Accuracy arguments are the en vogue route in epistemic justifications of probabilism and further norms governing rational belief. These arguments often depend on the fact that the employed inaccuracy measure is strictly proper. I argue controversially that it is ill-advised to assume that the employed inaccuracy measures are strictly proper and that strictly proper statistical scoring rules are...
متن کامل